Permutation Diffusion Maps with Application to the Image Association Problem in Computer Vision

Consistently matching keypoints across images, and the related problem of finding clusters of nearby images, are critical components of various tasks in Computer Vision, including Structure from Motion (SfM). Unfortunately, occlusion and large repetitive structures tend to mislead most currently used matching algorithms, leading to characteristic pathologies in the final output. In this paper we propose a new method, Permutations Diffusion Maps (PDM), and a related new affinity measure, Permutation Diffusion Affinity (PDA), to solve this problem. PDM is inspired by Vector Diffusion Maps, recently introduced by Singer and Wu, and uses ideas from the theory of Fourier analysis on the symmetric group. We show that when dealing with difficult datasets, using PDM as a preprocessing step to existing SfM pipelines can significantly improve results.